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Abstract 

Virtual environments (VEs) should be able to provide experiences as rich and complex as those to be had in real life. 
While this seems obvious, it is not yet possible to create a perfect simulacrum of the real world, so such 
correspondence requires the development of design techniques by which VEs can be made to appear more real. It 
also requires evaluation studies to determine if such techniques produce the desired results. 

As emotions are implicated in our phenomenological understanding of the physical world, they should also play an 
integral role in the experience of the virtual one. Therefore, a logical sequence of experimentation to understand how 
VEs can be made to function as emotion-induction systems is in order. The Sensory Environments Evaluation (SEE) 
research program has developed a twofold design process to explore if we react to virtually supplied stimuli as we 
do to the real world equivalents. We look at manipulating both the sensory and emotional aspects of not only the 
environment but also the participant. We do this with the focus on what emotional affordances this manipulation 
will provide. Our first evaluation scenario, DarkCon , was designed in this way to produce a strong sense of 
presence. Sixty-four subjects have been fielded to date and the data is currently being analyzed for results. We hope 
to find that rich design techniques along with the frame of mind with which a VR experience is approached will 
predictably influence perception and behavior within a virtual world. We will use these results to inform continuing 
research into the creation of more emotionally affective VEs. 

Keywords: Immersive Environments, Virtual Environments, VEs, Virtual Reality, emotion, affordance, fidelity, presence, 
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1 Introduction 

It has long been posited that virtual environments (VEs) can substitute for real world experiences. It is only recently, 
however, that the VR community has begun to focus on research that correlates the VR experience to exacting 
measures that can corroborate this assumption. In 2002, the University of North Carolina (UNC) took the first 
objective step in testing a virtual environment for its emotional equivalence to a real world experience. Starting with 
an extremely well known arousal stimulus (a specific one that our brain wiring has built up a predictable response to 
over our evolutionary past as demonstrated in Walk and Gibson’s classic experiment with babies) (Walk & Gibson, 
1961) Fred Brooks and his research group constructed a virtual world to replicate these real world stimulus 
conditions. Subjects were exposed to a deep pit that appeared to be a space into which they could fall. In real life, 
exposure to such a space (whether real or painted as a trompe-l’oeil), results in increased heart rate (HR) and skin 
conductance response (SCR). UNC’s study proved that their VE induced physiological responses were in alignment 
to those generated by the real world correlate. (Meehan, Insko, Whitton & Brooks, 2002) 

The Sensory Environments Evaluation (SEE) project at USC’s Institute for Creative Technologies has taken the next 
step: determining if it is possible to obtain a similar measure of response to stimuli that do not have such predictable 
reaction patterns. To do this we created a VE designed with multi-sensory inputs and “hot spots” specifically 
crafted to produce emotional response. We took special care in the design process to set up situations and elements 
that presented opportunities for greater connection to the virtual experience, leading, we hypothesize, to greater 
sense of Presence, which is a topic of intense discussion in the VR community today (Ijsselsteijn & Riva, 2003). 
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We then tested sixty-four subjects within this environment, recording their biometric and behavioral data throughout 
to see if responses correlated to our expectations. 

Our ultimate goal in this research is to determine whether a VR environment can be reliably used as a viable 
substitution for real world experience, to the level of producing comparable biographic memories via the virtual 
emotional stimuli. In other words, can virtually induced emotional memory form similar neurological bindings as 
occur in real life? If we can prove this does happen, the implications for VR in the areas of education, training, 
entertainment, healing and personal growth may be profound. 

2 The role of emotions within VEs 

Emotions are implicated in our phenomenological understanding of the physical world, and therefore we surmise 
they play the same integral role in our experience of a virtual one. In addition, an abundance of recent research 
points empirically to the interconnectedness of emotions, cognition, learning and behavior, indicating that emotions 
cannot be easily dismissed when focusing on pedagogical goals. Emotions have been shown by Lane and colleagues 
(Lane, Chua & Dolan, 1999) to heighten the attention and response to visual cues, and by implication this may apply 
to other senses as well. Emotions provide the affective valence to arousal states and contribute to the qualia, or 
subjective phenomenal experiences, of the virtual environment. They are also integral to the formation of many 
desired affordances the VE should include if it is to possess a high degree of presence. A logical sequence of 
experimentation is therefore indicated to understand how/if VEs can be made to function as effectual emotion- 
induction systems. Towards this end, the SEE project is investigating several aspects of the human participants’ 
emotional state within virtual environments used for training. 

We would ultimately like to empirically measure emotional responses of the VE participants. However, even the 
most sophisticated biometric equipment (functional magnetic resonance imaging or fMRI and Near Infrared 
Spectroscopy, or NIRS) can do little more than measure increased blood flow in specific brain regions, as described 
in (Hoshi, Chen & Tamura, 2001). As such systems are also costly, we have utilized instead HR and SCR 
biometrics, mentioned previously, that record arousal states. While these measures may not tell us the specific 
emotions being experienced, they do provide a reaction status that can be correlated in time to our designed 
emotional triggers. 

Our approach is threefold. First we look at the design process through the various sensory inputs available to the VR 
practitioner (sight, sound, smell, haptics) and determine how these can be effectively combined to elicit emotional 
responses from the participant. Secondly, we examine how emotions provide affordances within the VE that 
enhance feelings of presence and connection. Thirdly, with our findings from the first two processes, we plan to 
devise a methodology for using emotions to increase the effectiveness of virtual environments for a variety of 
purposes. This paper focuses on the first two aspects of our approach, and then describes a study that explores these 
areas. 

2.1 Affective Virtual Environment Design 

The affective nature of a VE must be integrated into the design thinking at the earliest stages. As important as 
devising a traditional map of the space to be built is formulating the emotional map of this same territory. Space, 
virtual or otherwise, is not an entirely neutral container; it carries within it the basis for how people will respond to 
it. Of course, equally as important is what the person entering the space brings to it. It is up to the designer to 
manipulate both these aspects: the space and the person who will experience it. We will next describe our twofold 
design process that takes into account both these perspectives. 

2.1.1 Setting up the environment 

2.1.1.1 Tools and approach 

The tools of the immersive environment designer include (to date and with varying degrees of fidelity) four of the 
five basic sensory inputs: sight, sound, smell and haptics. It is somewhat inaccurate to think of these as digital 
reconstructions of “real” sensory phenomena, as their ultimate effect is to send the same physical signals to our 
neuronal pathways. Digital graphics, through various delivery modalities, are still output as photons that travel to 



our visual cortex through our optic nerve. Digital sounds, through the medium of air, still produce waves that cause 
vibrations in our otic mechanisms, and haptics rely on stimulating touch receptors throughout our bodies. Smells are 
a rather unique sensory system. Being biochemical in nature, they rely on actual molecules entering our nasal 
epithelium and binding to odorant receptors and therefore these are not susceptible to digital encoding. (Buck & 
Axel, 1991) 

Computer-rendered visuals can be created in a simple form, providing the minimum functionality to represent a 
specific or idealized space. While sometimes such minimalism, or abstraction, is desired because it is critical to the 
goals of the VE, it is not the way the real world works. Our visual world is a messy conglomeration of colors, 
textures, and objects in rich profusion. Such richness constitutes corroborative detail that we, as humans have been 
conditioned to expect. In a realistic immersive world, which aims to replicate space as the reality with which we are 
familiar, any lack of this detail may be perceived as fake or contrived. 

Just as visual richness is essential to our phenomenological knowledge, so too is our multisensory experience. We 
rarely perceive the world one sense at a time. All are orchestrated in a rich perceptual soup/concert. Our brain has 
evolved to deal with precisely this type of manifold experience. Each of our neurons possesses upwards of 10,000 
connections. Neurons in the VI region of the brain (our first tier vision area), for example, are also connected to a 
multitude of other sensory regions through those connections. (Felleman & Van Essen, 1991) This recent 
neurological understanding has contributed to our methodology of looking at each world element as part of an 
integrated sensory package. The effect of this approach is to bring about a veridical space that essentially “feels 
real’', even if no one of its sensory elements is perfectly realistic. 

2.1.1.2 A typical design practice 

The SEE Project’s first experimental world is DarkCon - so called because it emulates a nighttime reconnaissance 
mission with the participant in the role of a scout who must make a determination of potential danger from 
inhabitants of an abandoned building complex near a river. The choice of nighttime was made explicitly to impose 
difficulties (obscurity, concealment, etc.) on the mission’s observational tasks, as well as to elicit a sense of unease 
based on these difficulties. 

We began the design process by considering each element of the virtual world through its denotive and connotative 
sights, sounds and smells. For example, the first segment of the user’s journey occurs in a dilapidated culvert. The 
team members were asked to place themselves in the culvert mentally and describe it in evocative sensory terms, 
answering the questions: What is in the culvert? What noises do you hear? What does it smell like? What has it been 
used for? 

We collectively imagined the culvert with mossy, crumbling brick walls, a trickle of sluggish water running down its 
center, edged with fetid muck. There would be detritus too, flotsam carried by the rushing waters of the rainy season 
and deposited here: bits of tree limbs, old bike wheels, a rubble of rocks and miscellaneous indeterminate garbage. 
We conjectured elements of a waterworks infrastructure: hissing pipes, thumping generators that powered sputtering 
red lights, seeping spigots and shadowy alcoves. What creatures might populate such a culvert? Small creatures like 
bats might hang from the ceiling pipes, dropping pungent guano into the mud, and rats would most certainly skitter 
amongst the debris searching for scraps. Humans have left their mark in this culvert too: bits of cultural refuse: a 
worn suitcase, its contents spilled out into the grunge, empty tin cans and abandoned water bottles, cigarette butts 
here and a photo album with forgotten faces there, all left to the fate of the elements. 

Now to make it more evocative and heighten the “feels real’’ sense: the bats should be disturbed if one passes too 
near; the rats scatter to hiding places; there could be evidence of violence: blood stains on the walls, bullet shells in 
the mud. The culvert itself should add a sonorous reverb to the sounds of trucks and other vehicles traveling on the 
road above. This shakes loose bit of wall and rock, with the effect of startling anyone nearby. Mysterious creaks and 
rumbles that cannot be easily identified punctuate the aural curtain, as do sounds from the outside, distant trains and 
dogs barking, a barely perceptible gurgle of a river and chirping crickets. In this way we create the world in our 
mind, and recreate it in the sensory aspects of the virtual world. 

We use standard tools to create the visuals and sounds conjured by the descriptions we have formed; 3D modelling, 
texturing and animation software, and digital sound mixers. Smells are not so straightforward. As there are no 
simple primitives for smells (as there are for color, in red, green and blue), each odor in the world is a complex and 



distinct scent. We have devised a prototype collar to hold four unique scents that can be triggered individually to 
release at specific locations in the virtual terrain. (Later versions will include up to ten unique scents.) We also use a 
special form of passive haptics provided by an infrasound generator that can serve as an almost imperceptible 
“soundtrack” for the experience. Infrasound serves augment feelings of unease felt by a participant so its judicious 
use in the VE can provide a sense of anxiousness where needed. (Backteman, Koehler & Sjoberg, 1983) 



Figure 1: The ICT scent collar Figure 2: Bats in the culvert 


Such attention to the “story” of the place through its corroborative detail and sensory orchestrations contributes 
immensely to the phenomenological realism of the VE. It provides expected details that concur with real world 
experience and creates the potential for a powerful sense of presence. 

2.1.2 Setting up the participant 

An affective VE does not work in isolation: it is a complimentary dialogue between environment and user. The 
participant is entitled to some understanding and setup for what he or she is about to undergo, or their experience 
may be ambiguous. This is an often-neglected aspect of virtual world design. A subject is typically told nothing 
(beyond a cursory introduction to how the VR accoutrements work), or exactly what the world is supposed to do 
to/for them through verbal or printed commentary. In our view, it is preferable to establish a connection to the VE 
for the participant that allows the dialogue to flow naturally from the individual’s innate reactions. To achieve this 
we settled on the mechanism of psychological priming. (Tulving & Schacter, 1990) (Schacter & Buckner, 1998) Our 
priming consists of a short video clip that precedes entry to the virtual world. The video ostensibly serves to brief 
the participant on a mission that must be accomplished during their time within the VE. But construction of the 
priming accomplishes several more subtle but critical factors: 

• It sets up a vehicle for a context-driven emotional connection by establishing a backstory. 

• It serves to construct a schema for user experience 

• It supplies reference points that allow a user to make associations and fill in the gaps as the experience 

unfolds. 

• It invokes motivation by providing a goal to accomplish. 

• It allows for dissemination of both overt and subliminal cues that can steer subsequent interpretation of the 

VE objects and events 

• It lends more substance to the VE making it more convincing as a total experience. 

When devising the priming video we take into account not only the words that are spoken, but also how they are 
delivered. The tone of voice, who is on camera, what persona is projected, the costume that is worn, and the setting 
they are in, are all clues that users will internalize, whether they are aware of them or not. Because we feel these 
aspects to be critical to the success of the experience, we chose to concentrate our first evaluation efforts on 
determining the effect such priming has on the user’s behavior and arousal states while in the VE. 




2.2 Affordances provided by our design process 


The term affordances was first defined by perceptual scientist James J. Gibson (1979) to refer to the connection 
between perceptual elements in the environment and the possibilities for action they present to the human perceiver. 
Our work builds on Gibson’s affordances by expanding the concept to include a wide range of affective elements 
that provide opportunities for emotional reaction. Affective elements require a broader definition of perception 
because they may fall below the levels of conscious perception and still afford mental and physical reactions. We 
use the term emotional affordance to refer to this concept and consider it to be a critical part of the VE design 
process. If perceptual affordance is a perceptual cue to the function of an object that causes an action, then 
emotional affordance is a sensory cue to the function of a stimulus that causes an emotional reaction. 

Designed emotional affordances must seem natural to the situation. They can serve several purposes, including 
subtly guiding the participant along a desired path through the VE. Clive Fencott (Fencott, 1999) describes 
affordances in terms of three basic perceptual opportunities - sureties, shocks and surprises - that could be designed 
in a VE. He breaks down these opportunities further into attractors, connectors and retainers. This is similar to our 
work in what we term coercive narrative (Morie, 2001) that includes not only the perceptual elements Fencott 
outlines, but also subliminal and emotional components that contribute to what we see as a continuum of affordance. 
We propose this continuum of affordance as complementary and overlapping realms of perception AND emotion. 
All affordances are user contingent; in a VE they are essentially triggers that result in an action (physical response) 
or a reaction (emotional response) from the participant. 

Continuum of affordances 

action „__ reaction 



Perceptual Emotional 


Figure 3: The affordance continuum 

There are a great many examples of these affordances in the DarkCon scenario and we will describe a few. First, the 
path of the culvert in which the journey starts limits the participant’s movements, requiring them to traverse the 
tunnel (spatial surety or coercive narrative). The human refuse evokes questions of who might have left these things 
and what the people were doing here (associate memory, which helps build a more detailed story about the place). 
The small creatures in a dark and unknown space heighten the sense of unease, as do the unidentified sounds (is 
someone sneaking up on me?) and the rumbles from the trucks passing overhead (I won’t be able to hear if someone 
else is in here). These last two sensory cues not only elicit fear, they also tend to lower an emotional threshold that 
may cause other stimuli to have effects they might not have in more familiar emotional territory. In addition, use of 
the infrasound frequencies will tend to cause people to avoid its presence if possible, and thus affects movement (as 
a detractor) that is not only subliminal but that might not otherwise seem warranted. We use perceptual cues 
(attractors) such as loud sound and flashing lights to attract the subject’s gaze towards an important event in the 
landscape. We relay on startle mechanisms (Fencott’s shocks) where participants disturb the bat colony causing 
several bats to speed out into the night. This not only increases the aversion to the culvert space, but also points the 
way out of the space. (If the bats can go that way then so can I.) 

2.2.1 From design to evaluation 

Now that we have our world designed and built as affectively as possible, and have provided the basis for a schema 
through which the world is to be experienced, we are ready to use it for our initial studies. Despite the broadness of 
the SEE Project’s ongoing interests, we limited ourselves to first focus on the aspect of priming to see if variability 
here would provide significant differences in a subject’s behavior and apprehension of meaning ascribed to the VE 
experience. 






























3 Experiment Design Process 


For this first study we tested 64 subjects ages 18-40 drawn from a standard university population, divided evenly 
across two groups. The first group receives priming that indicates they will be experiencing an entertaining video 
game, and the other group is made to believe that they will be entering a serious military training exercise. 

3.1 Evaluation Procedure 

Our experiment starts with standard pre-experience including a modified Immersive Tendencies Questionnaire, or 
ITQ (Witmer & Singer, 1998), the Simulator Sickness Questionnaire, or SSQ (Kennedy, Lane, Berbaum & 
Lilienthal, 1993) and the Virtual Environments Questionnaire, VEQ (Usoh, Catena, Arman &Slater, 2000) Most 
important among the modifications we have made in the ITQ are questions that detail a participant’s history with 
various types of video games. We thus can compare game players and non-game players to see if this subject 
condition has any bearing on the outcomes of primed behavior observed in the training simulation. When 
participants are brought into the room to fill out these questionnaires, we have already started the priming process. 
Themed posters decorate the walls of the room: game-based for the game primed participants and serious military 
posters for those undergoing the second form of priming. These are not referred to but form a backdrop in the room 
in which participants fill out their forms. In fact, if asked about them, we say we cannot comment about anything in 
the room. 

Subjects are then outfitted with physiological monitors that record HR and SCR and afterwards brought into ICT’s 
Virtual Reality Theater where they are fitted with the VR viewing and navigation equipment. To minimize any 
unfamiliarity with the VR devices they must use (an InterSense acoustic and inertial tracker, with X and Y 
dimensions mapped onto a joy stick, and Z, yaw and pitch mapped onto a sensor mounted on the head mounted 
display, or HMD) we let them practice in a VE “Tutorial Room”. This room is architecturally and emotionally 
neutral, but allows a person to experience the range of movements and observational skills required within the 3D 
evaluation scenario. We found this to be a necessary step as, absent such familiarization, participants often perform 
with a very limited range of movement, such as not crouching or bending down, and looking only straight ahead 
with “tunnel vision”. 

Once the participant has gained adequate familiarity within the tutorial room (determined by the evaluator and 
subject together), we record a final baseline for the physiological data, which allow us to compensate for any impact 
the overhead of using the VR equipment might induce. It is now time for the experiment to begin. Still in the HMD, 
the participant is shown one of the two video clips that outlines the mission he or she will undertake in the DarkCon 
VE. 

Each version of this video reflects one of the two priming conditions. Each two-minute clip is delivered by an 
appropriately themed narrator (one dressed comfortably in civilian clothes and one in military uniform), in an 
appropriately themed environment (a den lined with books or a room with military maps on the walls, respectively). 
While the specific details of the mission are consistent between the two briefings, the beginning and ending 
segments of the scripts vary to support the priming goals. 



Figure 4: Themed actors deliver the priming video 


Figure 5: One of the ending scenes-dog attack 






The mission itself consists of a basic observational exercise to determine if suspected rebels inhabit the area, and a 
simple task of dropping a GPS transmitter near a specific building if enough clues indicate that it is. The use of a 
civilian subject pool required us to describe the mission in language that civilians could understand. An early 
version of the mission briefing, written with military subject matter experts, turned out to be too difficult for 
civilians to grasp. 

At the close of the priming video the DarkCon experience comes into view in the stereo HMD. Given their mission 
briefing instructions, participants know what they need to do and where they need to go in the virtual environment. 
Now, they may choose to follow the mission to the best of their ability, or they may not. Because of the free will 
given to the participants, in essence each experience is unique. Not having commensurate experiences may appear 
to be a problem when embarking on studies in VE, but it is this agency that is precisely why we are interested in 
such experiences. Each person’s choices and decisions within the unscripted world are their own. Such choice is 
not only necessary for this particular study, but has rich implications for individualized training or education that 
includes cognitive decision-making. The only exception to this free agency is that we have made it virtually 
impossible to complete the mission without being discovered. There are several variations on this discovery (for 
example, vicious dogs, landmines, being shot) but all are designed to elicit strong arousal states. 

4 Data mining in multi-variate dataspace 

Because of the participant’s agency and the fact that we are utilizing several types of data collection, both qualitative 
and quantitative, this study poses challenges in data analysis. Data includes subject characteristics from 
questionnaires (described previously) administered both before and after the VE experience, behavioral and 
biometric data collected throughout the virtual environment experience, and recall and retention tests administered 
directly after the experience and one week later, respectively. While the pre and post instrumented tests are dealt 
with in more traditional ways, the quantitative recordings of the participant data has motivated development of new 
tools to more effectively deal with it. 

4.1 Visualization of behavior and arousal data 

We are evolving visualization techniques to measure behavior and correlate it to physiological responses. With a 
custom tool we call Phloem, we can investigate a full 6 DOF recording of all the participant’s movements (x, y 
location, z altitude for stoop/stand, and head orientation to ascertain what is being looked at) from the entirety of the 
experience and view this alongside the temporally associated physiological data. Phloem also allows queries to the 
stored data, such as how long a participant took to go through the world (or any segment thereof) as well as where 
and for how long they remained at any particular point. Did he or she spend a great deal of time in the tunnel, for 
example, indicating perhaps that the participant was taking greater care in his observations? For example, we may 
want to know how many times and for how long a participant observed a particular object of interest, say a machine 
gun, so that we can compare this to the number of times this object was reported, or to correlate whether the sighting 
created a increase in HR or SCR. (This is actually an accounting of how long the object was within the field of view 
of the participant. We can assume that if the object was in that FOV for a certain number of seconds, the chances 
that the object was seen by the participant are good.) 

The recorded simulation data can also be scrubbed over visually through a GUI that shows the values throughout the 
experience. As well, this data can drive a full sensory recreation of the scenario for further perceptual examination 
and correlation. Having this data allows us to intuitively associate specific behavioral aspects with the physiological 
data being captured throughout, which can then be compared to participant data from the questionnaires and self- 
reports. 



Phloem screens of a user experience 



Behavioral data Physiological data Visual playback 

Figure 6: Data as shown by the Phloem visualization tool 

4.2 Detailing participant response 

To determine the effect of our emotional affordance design we must look carefully at each subject’s personal 
experience. The Phloem tool is invaluable in this approach. We have divided the experience into epochs of interest, 
the first within the culvert and the second in the open terrain, looking at the stimulus response within these epochs. 
As an example we will detail one subject’s experience via his recorded skin conductance response, as displayed in 
Figure 7, below. With SCR a stimulus-related response is generally observed within l-4s after the stimulus occurs. 
For this reason a time scale of 20s with a minor of 4s is displayed to help read the annotated SCR graphs 
corresponding to the time spent inside the culvert (154.8s) and outside the culvert (266.4s) before this particular 
subject was detected by the roof guard/sentry. The unit for the SCR is lOmV/uS (10 millivolts per micro siemens), 
which is the gain used for this subject MN03184A. On the vertical axis 1 unit = O.luS. 

The first large SCR signal displays an anticipatory response, i.e. the subject had just started out in the dark tunnel, 
and was looking ahead. Fie sidestepped and walked along the wall towards the red glow of an alcove containing a 
pump, which was heard but not seen. Just as the participant reached the alcove, he turned his gaze to his right and 
down, noticing debris (in this case, a water carrier labeled Vodd). After a turn he finally made a visual contact with 
the pump, then looking back over to his left and down he noticed a baby doll in the mud. Note that when the 
participant looked at details (Voda and the debris) strong SCRs were elicited. The participant then moved on 
towards overhanging pipes and looked more closely at the pipes and bats hanging from them. A response from the 
bats flying away is observed (which we designed as a startle stimulus). As he continued down the side tunnel 
another response is observed. The participant saw and heard stones rolling down a small dirt mound while hearing a 
distant (truck) rumble, which produced two clear SCRs. 

At the culvert exit, but before leaving its confines, the participant looked outside (left and right), with an observable 
small cluster of SCRs and simultaneously, a pronounced drop in HR (not shown on this chart). This might reflect 
some an attentional orientation of the participant to the novel environment of the outside world. At the exit a loud 
truck was first heard, and then seen, with the subject’s gaze traveling upwards to the overhanging bridge, bringing 
the large truck into visual contact - both stimuli producing small responses. It is interesting to observe the responses 
when the main building, the wrecked car wreck and the damaged building are seen. Of great interest to our study is 
to notice the subject’s response when he first saw the lookout guard on the rooftop. It seems that there was a slight 
confusion in deciding where to go next and the greatest response appeared when the subject looked back at the 
culvert entrance. Did he feel exposed now to more immanent danger? Was he searching for a place to hide? A little 
later, this subject saw the roof guard again, and reacted by running for cover behind a wrecked car (placed there for 
exactly this purpose). At that point he was detected, and the end of the scenario was played. In this case the 
scenario was ended with a dog attack. Overall, this subject’s SCR indicates responses at many of our purposely 
placed emotional hotspots. 
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Figure 7. Annotated Skin Conductance results from one DarkCon subject obtained via a replaying of the experience 
with Phloem 


5 CONCLUSIONS 

We have found that subjects demonstrate a wide range of behaviors and responses. This is to be expected; it is 
precisely these individual manifestations of behaviours and responses within a free agency virtual environment that 
most interest us. While it took time to create the annotations you see here, such initial forays into this complex data 
space have shown us what is important to automate. For example, we can overlay the heart rate derived from the 
EKG measures onto the SCR diagrams. Other information can be extracted from the data and visualized via 
Phloem. We can determine, for instance, what the participant has likely seen by setting up queries to Phloem’s 
behavioral data to determine if, when, and for how long a stimulus object has been in a subject’s field of view (what 
we call looks). With that information we can then set up a graphic overlay of such looks with the SCR and HR data 
and easily find the correlations present per each subject’s unique experience. We eventually expect, with a large 
enough subject population, to be able to expose typical patterns of behavior within such free agency worlds. 

Ongoing work will result in even more robust tools to effectively deal with the multi-variate data from our virtual 
environment studies. The SEE Project is just beginning this long term program of research that we hope will benefit 
the entire community of virtual environment researchers. 
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